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Abstract 

In this paper, we develop a multistage approach for estimating the mean of a bounded 
variable. We first focus on the multistage estimation of a binomial parameter and then gener- 
alize the estimation methods to the case of general bounded random variables. A fundamental 
connection between a binomial parameter and the mean of a bounded variable is established. 
Our multistage estimation methods rigorously guarantee prescribed levels of precision and 
confidence. 

1 Introduction 

The estimation of the means of bounded random variables finds numerous applications in var- 
ious fields of sciences and engineering. In particular, Bernoulli random variables constitute an 
extremely important class of bounded variables, since the ubiquitous problem of estimating the 
probability of an event can be formulated as the estimation of the mean of a Bernoulli variable. In 
many applications, one needs to estimate a quantity /U which can be bounded in [0, 1] after proper 
operations of scaling and translation. A typical approach is to design an experiment that produces 
a random variable Z distributed in [0, 1] with expectation run the experiment independently a 
number of times, and use the average of the outcomes as the estimate [?]• This technique, referred 
to as Monte Carlo method, has been applied to tackle a wide range of difficult problems. 

Since the estimator of the mean of Z is obtained from finite samples of Z and is thus of random 
nature, for the estimator to be useful, it is necessary to ensure with a sufficiently high confidence 
that the estimation error is within certain margin. The well known Chernoff-Hoeffding bound 
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[3] [6] asserts that if the sample size is fixed and is greater than then, with probability at 
least 1 — S, the sample mean approximates fi with absolute error e. The problem with Chernoff- 
Hoeffding bound is that the resultant sample size can be extremely conservative if the value of 
/U is close to zero or one. In the case that fx is small, it is more reasonable to seek an (e, 5) 
approximation for in the sense that the relative error of the estimator is within a margin of 
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relative error e with probability at least 1 — 5. Since the mean value jj, is exactly what we want 
to estimate, it is usually not easy to obtain reasonably tight lower bound for /x. For a sampling 
scheme with fixed sample size, a loose lower bound of /i can lead to a very conservative sample 
size. For the most difficult and important case that no positive lower bound of ^ is available, it is 
not possible to guarantee prescribed relative precision and confidence level by a sampling scheme 
with a fixed sample size. This forces us to look at sampling methods with random sample sizes. 

The estimation techniques based on sampling schemes without fixed sample sizes have formed 
a rich branch of modern statistics under the heading of sequential estimation. Wald provided a 
brief introduction to this area in his seminal book [TU]. Ghosh et al. offered a comprehensive 
exposition in [5]. In particular, Nadas proposed in [9] a sequential sampling scheme for estimating 
mean values with relative precision. Nadas's sequential method requires no specific information 
on the mean value to be estimated. However, his sampling scheme is of asymptotic nature. The 
confidence requirement is guaranteed only as the margin of relative error e tends to 0, which 
implies that the actual sample size has to be infinity. This drawback severely circumvents the 
application of his sampling scheme. 

In this paper, we revisit the sequential estimation of means of random variables bounded 
in [0,1]. To overcome the limitations of existing methods, we have developed a new class of 
multistage sampling schemes. Our sampling schemes require no information of the unknown 
parameters and guarantees prescribed levels of precision and confidence. The remainder of the 
paper is organized as follows. Section 2 is devoted to the multistage estimation of a binomial 
parameter. In Section 3, we generalize the estimation methods of a binomial parameter to the 
mean of a bounded variable. In Section 4, we establish a link between a binomial parameter and 
the mean of a bounded variable. We demonstrate that the estimation methods for estimating a 
binomial parameter can be easily applied to the estimation of the mean of a bounded variable by 
virtue of this link. Section 5 is the conclusion. All proofs are given in the Appendices. 

Throughout this paper, we shall use the following notations. The expectation of a random 
variable is denoted by E[.]. The set of integers is denoted by Z. The set of positive integers is 
denoted by N. The ceiling function and fioor function are denoted respectively by [.] and [.J (i.e., 
\x\ represents the smallest integer no less than x; \x\ represents the largest integer no greater 
than x). The notation sgn(2;) denotes the sign function which assumes value 1 for x > 0, value 
for X = 0, and value — 1 for x < 0. We use the notation Pr{. | 6} to indicate that the associated 
random samples Xi, X2, • • • are parameterized by 0. The parameter 6 in Pr{. | 0} may be dropped 
whenever this can be done without introducing confusion. The other notations will be made clear 
as we proceed. 

2 Estimation of Binomial Parameters 

Let X be a Bernoulli random variable defined in a probability space (0, Pr) such that Pr{X = 
1} = 1 — Pr{X = 0} = p G (0, 1). It is a frequent problem to estimate the binomial parameter p 
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based on a sequence of i.i.d. random samples Xi,X2, • • • of X. To solve this problem, we shall 
develop multistage sampling schemes of the following general structure. The sampling process 
is divided into s stages. The continuation or termination of sampling is determined by decision 
variables. For each stage with index i, a decision variable Dg = &e{Xi, • • • , X^) is defined based 
on samples Xi, • • • , X^, where is the number of samples available at the i-th stage. It should 
be noted that ng can be a random number, depending on specific sampling schemes. The decision 
variable Dg assumes only two possible values 0, 1 with the notion that the sampling is continued 
until Dg = 1 for some i E {!,•■■ ys}. Since the sampling must be terminated at or before the 
s-th stage, it is required that Ds = 1. For simplicity of notations, we also define Dg = for £ = 
throughout the remainder of the paper. 

2.1 Control of Absolute Error 

In many situations, it is desirable to construct an estimator for p with guaranteed absolute 
precision and confidence level. For this purpose, we have 

Theorem 1 Let 0<e<i, 0<(5<1, C>0 '^''^-d p > 0. Let ni < n2 < ■ ■ ■ < Ug be 



the ascending arrangement of all distinct elements of 

'in— §— 
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i = 0, 1, • • • , r > with 



ln(l+p) 



i\Pe ~ l \ ~ t)^ — i + 2 in{cs) ' ^'^d Dg = otherwise. Suppose the stopping rule is that sampling is 
continued until Dg = 1 for some £ G {1, • " " > s}. Define p = — - where n is the sample size 
when the sampling is terminated. Define 

Then, a sufficient condition to guarantee Pr {\p — p\ <e\p}>l — 6 for any p G (0, 1) is that 

^Pr{p, >p + £, Df_i = 0, = l|p}<- VpSi?-, (1) 



For £ = 1, • • • ,s, define Kg = Xi, Pg = and Dg such that Dg = 1 if 



e=i 



J2MPi<P~e, De^i = 0, Dg = l\p} <- Vp e (2) 



e=i 



where both ^}) and (0j are satisfied if < ( < ^t:^ 



(r+l) ■ 



2.2 Control of Absolute and Relative Errors 



To construct an estimator satisfying a mixed criterion in terms of absolute and relative errors 
with a prescribed confidence level, we have 



Theorem 2 Let 0<(5<1, C>0 and p > 0. Let Sa and Sr be positive numbers such that 

3-2e„ 



< < I and rr^yf— < Er < 1. Let ni < n2 < ■ ■ ■ < Ug be the ascending arrangement of all 
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distinct elements of 
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For £=!,■■■ , s, define Ke = Pi 



4(3+5.) ^ J_ 
Ki 



i = 0, 1, • • • , T > with T = 



ln(l+p) 



6(l-£.)(3-e.) ln(C<?) 



4 + 21n(C5) Pe ^ 2{3-e^)^\n{CS)-9Tue'i 



1 - 2£ 

2 3, 

1 I 2p ' / 1 I "~ ^ ^ 6(l+s.)(3+£.)ln(Ca) 

2 ^ 3^'^ V 4 21n(C<5) ^ fl ^ 2(3+e.)2 ln(C5)-9nfe^ , 
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/or i = 1, ■ ■ ■ , s — 1 and Dg = 1. Suppose the stopping rule is that sampling is continued until 
De = 1 for some £ G {1, • • • , s}. Let p — ^'"^ where n is the sample size when the sampling is 
terminated. Define p* — ^ and 

= (j{^ + ^-^ (^'P*) --kelXu {p*} , ^- = IJ I A _ e (0,^-^) : fe e z| U {p*} , 



G(p*,l):fcGZ , 



T/ien, Pr I |p — p| < Eq or | < | p| > 1 - /or any p G (0, 1) provided that 
^Pr{p, >p + e„, De-i^O, Di^l\p}<- VpG^", 



€=1 



^Pr{p, <P~ea, , = 0, £>f = 1 I p} < - Vp € 



(3) 
(4) 
(5) 
(6) 



^ Pr{p, > p{l + Er), Df I = 0, = 1 I < - Vp € 
^ Pr{p, < p(l - Sr), Di_, = 0, Di = l\ p} < - Vp e 

£=1 

where these conditions are satisfied for < C < 2{t+i) • 
2.3 Control of Relative Error 

In many situations, it is desirable to design a sampling scheme to estimate p such that the 
estimator satisfies a relative error criterion with a prescribed confidence level. By virtue of the 
function 



7-1 
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1 / 7 



cxp 



7 



1 + e 



7-1 



1 / 7 



— e 



cxp 



7 



1 -£ 



1=0 \ / s / j^o 

we have developed a simple sampling scheme as described by the following theorem. 

Theorem 3 Let < e < 1, < 5 < 1, C>0 and p > 0. Let 71 < 72 < • • • < 7^ 6e 

the ascending arrangement of all distinct elements of | [§ (7 + l)] ^ 

ri„r3 1' 1 , iM1 v^n£ ^ 

with T = 



4(3+6) 1 1 
9e C<5 



i = 



ln(l+p) 



-Lei = '^^ — ^ where is i/ie minimum number of samples such that 
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, s, define Di such that Di 



and 



Di = otherwise. Suppose the stopping rule is that sampling is continued until = 1 for some 
£ G {!,•■■ Define estimator p = — - where n is the sample size when the sampling is 

terminated. Then, Pr | ^zii < e | p| > 1 — (5 /or any p G (0, 1) provided that C > is sufficiently 
small to guarantee g{e,js) < ^ o-nd 



ln(C(5) < 



4e2 



1 + e 



ln(l + e) 



^Pr{p, < (1 - = 0, - 1 I p} < - Vp G 



^ Pr{p, > (1 + s)p, De-i = 0, = 1 I p} < - Vp G ^+ 

£=1 

u;/iere ^+ = ULi {t^^) e (p*, 1) : m G n} anc? i?" = ULi {;;rfe G (p*, 1) : m e n} 
{0,Zs-i) denoting the unique number satisfying 



(7) 
(8) 
(9) 
G 



7£ 



where z£ = 1 + 



2(3+e)^ ln(C<5) 



£=1 

/or ^ 



2 ( ^ + 



2£^ 

3 



2p* 
3 



In this section, we have proposed a multistage inverse sampHng plan for estimating a binomial 
parameter, p, with relative precision. In some situations, the cost of sampling operation may be 
high since samples are obtained one by one when inverse sampling is involved. In view of this 
fact, it is desirable to develop multistage estimation methods without using inverse sampling. For 
this purpose, we have 

Theorem 4 Let 0<e<l, < 6 < I and ^ > 0. Let t be a positive integer. For £ = 1, 2, • • • , 



where ni is deterministic and stands for the sample size at the £-th stage. For 



, define such that = 1 ifp( > ^j^l^y) ut/l, ' -gf / '^^^ — otherwise, where 



let p£ = 

* — 1, 2, • • • , uejt7te 1^1 SUCH utau ±J (, = l ij pg ^ 2(ci+E)'Hn(C,St)-9ntei ' 

= ^ for 1 < ^ < T and 6e = 62'^~^ for £ > t. Suppose the stopping rule is that sampling is 
continued until = 1 for some stage with index £. Define estimator p = Pi, where I is the index 



of stage at which the sampling is terminated. Then, Pr{Z < oo} = 1 and Pr ■ 
for any p € (0, 1) provided that 2(r + l)( < 1 and inf 



£>0 ■ 



rii 



P-P 

P 



< £ 



> 0. 



\p} 



> 1 



2.4 Fixed- width Confidence Intervals 

In some literature, the estimation of p has been formulated as the problem of constructing a 
fixed-width confidence interval (L, U) such that U — L < 2e and that Pr {L <p<U\p}>l — 5 
for any p G (0, 1) with prescribed e G (0, |) and 5 G (0, 1). For completeness, we shall develop 
multistage sampling schemes in this setting. 

Making use of the Clopper- Pearson confidence interval we have established the following 
sampling scheme. 
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Theorem 5 For a G (0, 1) and integers < k < n, define 

' iffc = 



jC{n, k, a) 



p if A; > 



and Uin.k.a] 



1 \i k = n 
p if k < n 



with p G (0, 1) satisfying YJj=k " P)" ^ = % and p ^ (0, 1) satisfying J2j=o (j)p' (l " 

p-jn-j _ 2,ei (" > and p > 0. Let ni < n2 < ■ ■ ■ < Ug he the ascending arrangement 

of all distinct elements of 



ln(l+p) 



For 



: i = 0, 1, • • • , t| with T = 

i = 1, - ■ ■ ,s, define Ki = YllLi ^^^^ such that Di = 1 ifU{n£, K^, (6) — C{n£, K^, (6) < 2e; 
and Di = otherwise. Suppose the stopping rule is that sampling is continued until Di = 1 for 
some I G {I,-'" i^}- Define L = (n, X^f^i -'^i, C^) and U = U (n,J2f^^ XiX6) , where n is the 
sample size when the sampling is terminated. Define 

s s 

U {£(71,, k, CS) e (0, 1) : < fc < n,} , ^ \J Wine, k, (6) £ (0, 1) : < fc < n,} . 



Then, a sufficient condition to guarantee Pr {L < p < U \ p} > 1 — 5 for any p £ (0, 1) is that 



^PT{C{ne,KeXS)>P, -D^-i = 0, De = l\p}<- Wp€J^L, 



Mi^irii, Ke, CS) < P, Df ^ = 0, = 1 | p} < - Vp € 



(10) 
(11) 



where both /110\) and ill]) are satisfied if < ( < 2{t+i) ■ 

Making use of Chernoff-Hoeffding inequalities (Sj [6] , we have estabUshed the following sampling 
scheme. 



Theorem 6 For a G (0, 1) and integers < k < n, define 

(p for < A; < n, 

(f)" forA; = n, U{n,k,a) = < 

for k = 



I -it 



for < A; < n, 
for A; = 0, 
for k = n 

ln(C5) 



with p G (0, ^) satisfying Jt^ = and p G (|, 1) satisfying = where 

^b(-) ■) ^-5 a function such that ^-^{z^ 6) = zln | + (1 — z) In j^f for z G (0, 1) and 9 G (0, 1). Let 
C > and p > 0. Let ui < n2 < ■ ■ ■ < be the ascending arrangement of all distinct elements 



of 



( 2^- 



ln(l+p) 



For £ = 1, • • • ,s, define Ki 



i = 0, 1, • • • , r| with r 

Yl7=i-^i '^^^ such that Di = 1 ifhl{ni,Ki,C,5) — C{ni, Ki,CS) < 2e; and Di = otherwise. 
Suppose the stopping rule is that sampling is continued until = 1 for some i G {I,-'' 
Define L = ^ {'"■,J2"=i ^iXS) and U = U (n, X]"^! C<^)' where n is the sample size when the 
sampling is terminated. Define 

s s 

^L^\J {C{n(, k, (S) e (0,1) ■.0< k <ne} , = [j {U{nt, k, (6) G (0, 1) : < fc < n^} . 
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Then, a sufficient condition to guarantee Pr {L <p<U\p}>\ — 6 for any p G (0, 1) is that 



J2 M^ne, Ki, Q5) > p, De^, 0, Z), = 1 | p} < - e ^l, 

where both i f71|) and i flgj) are satisfied if < ( < 2{t+i) ■ 



(12) 
(13) 



Making use of Massart's inequality [8], we have established the following sampling scheme. 
Theorem 7 For a £ (0, 1) and integers < k < n, define 



k 3 

£(rt, k, a) ~ max ■^0, — I — 
' n 4 



2fe _ /TTZE^ -)1 

n y 2 In ^ ^ n' 



9n 
iln i 



r 



and 



k 31-1^ + 1/1 + 21^^(1-^) 



Zi(n, fc, a) = min < 1, h - 



1 



8 In - 



Let C > and p > 0. Let ni < n2 < ■ ■ ■ < Ug he the ascending arrangement of all distinct 

ln(l+p) 



elements o/|^|(^ + i)^ (^_i)in^j :« = 0,1,--- ,t} 
define Ki = X]r=i ^« ^'^^ swc/i t/iat = ^ if 



with 



For 



1, • • • 



1 



9 



21n(C(^) 



1 



L<e 



3ni 



3 21n(C(5) 



and Z?£ = otherwise. Suppose the stopping rule is that sampling is continued until = 1 for 
some I G {I,-"" i-?}- Define L = C (n,J2"^j^Xi,(S) and U ~ U (n,J2"^^XiXS), where n is the 
sample size when the sampling is terminated. Define 

s s 

^L=[j {C{n(, k, CS) e (0, 1) : < fc < , ^ [j {U{ni, k, (6) G (0, 1) : < fc < ne} . 

Then, a sufficient condition to guarantee Pr {Z < p <U \p^ > \ — b for any p £ (0, 1) is that 



J2 M^ini, Ki, C6) > p, Di_i = 0, = 1 I p} < - Vp e i^L, 



^ Pv{Uine, Ke, CS) < p, De , = 0, = 1 | p} < - Vp € 



(14) 
(15) 



where both ( [7^ and ( flJj) are satisfied if < ( < ^r^q 



(r+i) ■ 



It should be noted that the interval estimation methods described in Theorems [5HZ1 can be 
made less conservative by using tight bounds of C{p,e) = 1 — Pr{i < p < U \ p} ioi p £ [a,b] C 
in Theorem [SI Based on such bounds, a branch-and-bound type strategy described in section 2.8 
of [1] can be used to facilitate the search of an appropriate value of C such that the coverage 
probability associated with interval {L, U) is no less than 1 — 6. 
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Theorem 8 Let Ci = C{pi, (, S) and Ue = U{Pi, C, S) for £ = I, - ■ ■ ,s. Then, 
C{p, e) < Pr{L >a\b} + Pt{U < b \ a} 

s s 

< Y.^''^^^ ^ ^^-^ = 0, De = l\b} + ^Pr{Z^f < b, De_i = 0, De = l\ a}, 

1=1 e=i 

C{p, e) > Fv{L >b\a} + Fv{U < a \ b} 

s s 

> ^Pr{£^ > b, Di^i = 0, De = l\a} + < a, £>^-i = 0, De = l\b} 

for any p G [a,b]. Moreover, if the open interval {a,b) contains no element of the supports of L 
and U, then 

C{p, e) < Pt{L >b\b} + Fv{U <a\a} 

s s 

< ^Pr{£^ > b, De-i = 0, De = l\b} + ^Pr{Z^^ < a, D^^i = 0, = 1 | a}, 
1=1 1=1 

C{p, e) > Pi{L >a\a} + Pr{U < b \ b} 

s s 

> > «' -^^-1 = 0, De = l\a} + Y, M^^e < b, £>^_i = 0, De = l\b} 
i=\ e=i 

for any p £ (a, b). 

We would like to note that Theorems 1 and 2 of [1] play important roles in the establishment 
of the theorems in this section. As can be seen from Theorems 1-6, the confidence requirements 
can be satisfied by choosing to be sufficiently small. The application of the double-decision- 
variable method and the single-decision- variable method is obvious. To determine <^ as large as 
possible and thus make the sampling schemes most efhcient, the computational techniques such 
as bisection confidence tuning, domain truncation, triangular partition developed in [1] can be 
applied. 

With regard to the tightness of the double-decision-variable method, we can develop results 
similar to Theorems 13, 18 and 23 of [1]. 

With regard to the asymptotic performance of our sampling schemes, we can develop results 
similar to Theorems 14, 19 and 24 of [T]. 

3 Estimation of Bounded- variable Means 

The method proposed for estimating binomial parameters can be generalized for estimating means 
of random variables bounded in interval [0, 1]. Formally, let Z £ [0, 1] be a random variable with 
expectation p = E[Z]. We can estimate p based on i.i.d. random samples Zi,Z2,--- of Z by 
virtue of the following results. 
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Theorem 9 Let < e < ^ and < 6 < 1. Let ni < 112 < ■ ■ ■ < Ug be a sequence of sample 
sizes such that Ug > ^p-- Define fig = — ^ — - for I = 1, • • • ,s. Suppose the stopping rule is 
that sampling is continued until — ^| ^ ^) ^ j ^ 2 in(2s/(5) /^'^ some £ G {1, • • • ,s}. Define 
fi = — - where n is the sample size when the sampling is terminated. Then, Pr — /i| < e} > 
1-6. 



This theorem can be shown by a variation of the argument for Theorem 1. 



Theorem 10 Let < 5 < I, 0<ea<| and 

sequence of sample sizes such that Us > 2 (^-^ + ^ 
i = 1, - ■ ■ ,s. Define 



6ea 



3-2e„ 



< e,. < 1- Let ni < n2 < 



z:-i-5)ln(f)- Define fli 



for 5 - l^a - 



r^cel ^ -r. ^ 6(l-6^)(3-6^)ln(C6) 

21n(C5) ^ ^ 2(3-er)^ ln{CS)-9nee'i ^'^ 

niel ^ - ^ 6(l+e^)(3+£^)ln(C.5) 

21n(C<5) f^e 2(3+er)2 1n(C<5)-9"i!e? ' 



< Us be a 
^'-1 ' for 



1 else 



for i = 1, • • • , s — 1 and Dg = 1. Suppose the stopping rule is that sampling is continued until 
Di = 1 for some £ G {1, • • • , s}. Define /x = — - where n is the sample size when the sampling 
is terminated. Then, Pr{|/I — /i| < Sa or |/i — /i| < Er^} > 1 — 6. 



This theorem can be shown by a variation of the argument for Theorem 2. In the general case 
that Z is a random variable bounded in [a,b], it is useful to estimate the mean fi = E[Z] based 
on i.i.d. samples of Z with a mixed criterion. For this purpose, we shall introduce the function 



, I „/-^7r-TT7Wr-T-rT for < z < 1 and G (0, 1), 

M{z,^) = 3 +3A 3 +3 ^) 

-00 for < z < 1 and /.i ^ (0, 1) 

and propose the following multistage estimation method. 

Theorem 11 Let Q < 5 < 1, > and < e,. < 1. Let ni < n2 < ■ ■ ■ < Ug be a sequence of 
sample sizes such that Ug > ^ In (^). Define /x^ = — ^, fJ-i = a + ^^/i^, 

tf = a+7 mm <^ fig - Ea, — ,^ . > , fJ-i = a+- max <^ /i^ + e^, \ 

b - a [ 1 + sgn(/2^)er J b- a [ 1 - sgn{fX(^)er J 

fori = !,••• ,s. Suppose the stopping rule is that sampling is continued until M(Jl£, fi^) < ^ In ^ 
and Ai{fig, /x^) < — In 2^ for some £ G {1, • • • ,s}. Define p, = — - where n is the sample size 
when the sampling is terminated. Then, Pr — /i| < Ea or \fl — fi\ < er|/^f|} >1 — 5. 
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4 A Link between Binomial and Bounded Variables 



There exists an inherent connection between a binomial parameter and the mean of a bounded 
variable. In this regard, we have 

Theorem 12 Let Z be a random variable bounded in [0, 1]. Let U a random variable uniformly 
distributed over [0,1]. Suppose Z and U are independent. Then, 

E[Z] = Pr{Z > U}. 

Proof. Let Fz^u be the joint distribution of Z and U. Let Fz be the cumulative distribution 
function of Z. Since Z and U are independent, using Riemann-Stieltjes integration, we have 

pl pz pi pz pi 

Pr{Z >U}= / dFz,u = / dudFz= z dFz = E[Z]. 

J2=0 Ju=Q ' Jz=0 Ju=0 Jz=0 

□ 

To see why Theorem 1121 reveals a relationship between the mean of a bounded variable and a 
binomial parameter, we define 

1 for Z>U, 
otherwise. 



X 



Then, by Theorem US we have Fr{X = 1} = 1 - Fr{X = 0} = E[Z]. This imphes that X is 
a Bernoulli random variable and E[Z] is actually a binomial parameter. As a consequence, the 
techniques of estimating a binomial parameter can be useful for estimating the mean of a bounded 
variable. Specially, for a sequence of i.i.d. random samples Zi,Z2,--- of bounded variable Z 
and a sequence of i.i.d. random samples Ui, U2, • • • of uniform variable U such that that Zi is 
independent with Ui for all i, we can define a sequence of i.i.d. random samples Xi,X2, • • • of 
Bernoulli random variable X by 

^1 for Zi>U^, 
otherwise. 

V 

5 Conclusion 

We have established a new multistage approach for estimating the mean of a bounded variable. 
Our approach can provide an estimator for the unknown mean which rigorously guarantees pre- 
scribed levels of precision and confidence. Our approach is also very flexible in the sense that the 
precision can be expressed in terms of different types of margins of errors. 



X,, 
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A Preliminary Results for Proofs of Theorems 

We need some preliminary results, especially some properties of fmiction A4{z,fj,) defined in 
Section 3. 

Lemma 1 A4{z, z+e) is monotonically increasing with respect to z ^ (0, \—'^), md is monotoni- 
cally decreasing with respect to z ^ (l~T' ^~^)- Similarly, M.{z, z—e) is monotonically increasing 
with respect to z G (e, ^ + and is monotonically decreasing with respect to z ^ {\ + ,1). 

Proof. The lemma can be established by checking the partial derivatives 
dM{z,z + e) ( I 2e 



dz [(. + f)(l-.-f)]n2 3 

dM{z,z-e) [ I 2e 



dz [(.-f)(l-.+ f)]n2 ■ 3 

□ 



Lemma 2 Let ^ < e < \. Then, M{z,z + e) > M{z,z - e) for z e [O, i] , and M{z,z + e) < 
M{z,z-e) for z £ 

Proof. By the definition of the function .), we have that M{z,fi) = —oo for z G [0, 1] and 
H ^ (0, 1). Hence, the lemma is trivially true for 0<z<eoTl — e<z<l. It remains to show 
the lemma for z £ (e, 1 — e). This can be accomplished by noting that 

2£^(1 — 2z) 

M{z,z + e) - M{z,z - e) - 



3(. + f) (l-.-f) (.-f) (l-. + f)- 
where the right-hand side is seen to be positive for z S (e, ^) and negative for z G 1 — e). □ 



Lemma 3 M yz, j^j > M \^z, j^j forO<z<l-e<l. 

Proof. It can be verified that 

^ \ ka( ^ ^ 2e^z{2-z) 
M z, \ - M[ z, ' - 



l + e) \'l-e) 3(l + f)[l-. + e(l-f)](l-f)[l-.-e(l-f)] 
from which it can be seen that M [z, > M i^z, for z £ (0, 1 — e). 

Lemma 4 M{fi — e, fi) < M{fi + e, fi) for 0<£<fi<l<l — s. 



□ 
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Proof. The lemma follows from the fact that 
M{^i - e, jj.) - M {i-i + e, i^i) 



3(/.-|)(l-^ + |) (/i+l) {i-n-D 
where the right-hand side is negative for 0<e<;U<^<l — e. 



□ 



Lemma 5 M (^z, is monotonically decreasing with respect to z G (0, 1). Similarly, M (^z, 
is monotonically decreasing with respect to z G (0, 1 — e). 



Proof. The lemma can be shown by verifying that 

—M Z, = 7 r X 

V 1 + 2(1 + 1) + + 



2ezl2 
3 J 



for z S (0, 1) and that 



I z, 

oz 



1-e 



1-^/ 2(1-1) 



3 J 



< 



< 



for z e (0, 1 - e). 



□ 



Lemma 6 For any fixed z G (0, 1), M{z,fi) is monotonically increasing with respect to fi £ {0,z), 
and is monotonically decreasing with respect to fi £ (-^il)- Similarly, for any fixed fi £ (0,1), 
M{z,^) is monotonically increasing with respect to z £ (0, /x), and is monotonically decreasing 
with respect to z £ {fj,, 1). 

Proof. The lemma can be shown by checking the following partial derivatives: 



dM{z,ti) 
d/j, 

dM{z,ii) 

dz 



{z - li) -z)+ z{l - ^) + z{l - z)] 



2^ I z 
3 3 



1 _ 2i£ _ z 
-^3 3 



(/.-z) ^(1-f -|) + £^ 



+ 



2/£ 

3 



2/i , z 
3 3 



2/i 
3 



□ 



The following result, stated as Lemma [TJ is due to Massart [8]. 

Lemma 7 Let X„ = — - where Xi, ■ ■ ■ ,Xn are i.i.d. random variables such that < < 1 
and E[Xj] = n £ (0, 1) for i = 1, • • • , n. Then, Pr {X„ > z] < cxp {nM{z, ^)) for any z G (/x, 1). 
Similarly, Pr {X„ < z} < exp {nM{z, ^)) for any z £ (0,/i). 
Y^ri ^ 

Lemma 8 Let Xn = — - where Xi, • • • , X„ are i.i.d. random variables such that < < 1 



and E[Xi] = /U G (0, 1) for i = 1, 



,71. 



Then, Pr {X„ > ii, M (X„, a^) < ^} < a /or any a > 0. 
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Proof. Since the lemma is trivially true for a > 1, it remains to show it for a £ (0, 1). It can 
be checked that M{fi,i2) = 1 and M{^,^i) = Since ^Ml^ = (/z- z)[/i(l - f - f ) + ^]/ 

[(^ + f)(l ~ ~ f )]^ < for z G (/X, 1), we have that M.{z,fi) is monotonically decreasing from 
to 4(2m+'i) increases from to 1. To show the lemma, we need to consider three cases as 
follows. 

Case (i): ^2til+i) ^ ^^^^ case, we have that {X„ > /i, M (X„,^) < is an impossible 

event and the corresponding probability is 0. This is because the minimum of Ai{z,^) with 
respect to z G (fi, 1] is equal to Tr^—rr), which is greater than 



4(2^4+1) ' 

4(2^1+1) 

9(^-1) 



Case (ii): jrf^ = In this case, we have that Pt{X„ > ^i, M (X„,/i) < ^-^] = Vx{X, 



1, i = I,-- - ,n} = nr=iPi-{^» = 1} < nLiE[^»] = < cxp[nx ijy^j = a, where the last 
inequality is due to the fact that In fi < ^2i7+i) ■ Pi'ove this fact, we define g{n) = In/i — j^^^- 

Then, the first derivative of g{n) is g'{fi) = 4^^(2;l+'i/^~^'' - \il{2)Jrif'^ > for any ^ G (0, 1). 
This implies that g{ii) is monotonically increasing with respect to /i G (0, 1). By virtue of such 
monotonicity and the fact that g{\) = 0, we can conclude that g{^) < for any /i G (0, 1). This 
establishes Infi < ^2/7+1) • 

Case (iii): 4^2i7+i) ^ Ht- ^^^^ case, there exists a unique number z* G (/U, 1) such that 
M.{z*,n) = Since M.{z,^) is monotonically decreasing with respect to z G (/-f, 1), it must 
be true that any x G (/U, 1) satisfying /.i) < is no less than z* . This implies that 

{Xn > Ai, M (X„,/i) < ^} ^ {Xn > z*} and Pr{X„ > fi, M (X„,m) < ^} < P^' {^n > z*} < 
exp(n M{z*,fi)) = a, where the last inequality follows from Lemma [71 This completes the proof 
of the lemma. 

□ 



Lemma 9 Let Xn = ~' — - where Xi, ■ ■ ■ ,X„ are i.i.d. random variables such that < Xi < 1 
and E[Xi] = n e (0,1) for i = 1, ■■■ ,n. Then, Pr{X„ < /i, 7W (X„, /i) < ^-^} < a for any a > 0. 

Proof. Since the lemma is trivially true for a > 1, it suffices to show it for a G (0, 1). It can be 
checked that M{fi,i2) = 1 and M{0.^i) = j^^^- Since M^£:iii = (^^ - z)[(l - + f ) + t^]/ 
[(^ + f)(l ^ if ^ f)]^ > for z G (0, ;u), we have that Ai{z,fi) is monotonically increasing from 
4(2^^3) to as z increases from to fi. Now there are three cases: 

Case (i): 4^(^2il-3) ^ -'■^ ^^^^ case, we have that {X„ < fi, M (X„,^) < is an impossible 
event and the corresponding probability is 0. This is because the minimum of A4{z,fj,) with 
respect to z G [0, /i) is equal to 4(2^^3) 1 which is greater than 

Case (ii): ^^j^ = In this case, we have that Pr {X„ < ^, M (X„,/i) < ii^} = Pr{X, = 

0, * = I,... ,n} = m=lMx^ = 0} = nr=i(i - Mx^ ^ o}) < nr=i(i - m-^) = (i - ^^r < 

exp X 4(2^-3) ) = where the last inequality is due to the fact that ln(l - \i) < 4(277^- 
prove this fact, we define /i(^) = ln(l — fi) — j^^frgy- Then, the first derivative of is h'di) = 
"f^'+^^^r'^, < .^^""^^f'L < for any ^ G (0,1). This implies that h{i^L) is monotonically 

4(1 iijlzu o) 4(1 ll){2ii o) 
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decreasing with respect to ^ £ (0, 1). By virtue of such monotonicity and the fact that h{0) = 0, 
we can conclude that < for any /i G (0, 1). This establishes ln(l — n) < 4(2^-3) ■ 

Case (iii): 4(2^^3) < this case, there exists a unique number z* G (0,/i) such that 

A4{z*,^) = Since A4{z,fi) is monotonicahy increasing with respect to z € (0,/i), it 

must be true that any x G (0,/i) satisfying M.(^^ii) < is no greater than z* . This im- 
plies that {X„ <^l, M (Xn,fj) < } ^ {^n < and thus Pr{X„ < ^l, M (X„,m) < ^} < 
Pr {-'^ri < 2*} < exp(n7W(z*, /i)) = a , where the last inequality fohows from Lemma [71 This 
completes the proof of the lemma. 

□ 



B Proof of Theorem 1 

Throughout the proof of Theorem 1, we define random variables Dg, I = I,-- - ,s such that 
Dg = 1 if ~ 5I ^ - 3 + 2\l{cs) ^^'^ ~ otherwise. Then, the stopping rule can be 
restated as "sampling is continued until = 1 for some i G {1, • • • , s}". 



Lemma 10 = 1. 



Proof. 



By the definition of Ds, we have that {D,, = 1} = -\\-'^f>\ + it^}- 



the definition of sample sizes, we have 
Since {[\Ps-\ 



In ■ 



2e2 



> 0} is a sure event, it follows that ^{\Ps 



— which implies that 



- il - ¥r > 



1 I Us 
4 21n(C(5) 



< 0. 



2 ln(C5) 



} 



sure event and consequently Dg = 1. This completes the proof of the lemma. 



IS a 



□ 



Lemma 11 {p,<p- e, De = 1} C {p, < p, M {p„p) < '-^} for £ = I, ■ ■ ■ , s. 

> J + 2hi{cs) } ' suffices to show 



Proof. Since {D( = 1} = { - 5 

\pe<p-e, 



2 I 3 ) 



1 



— > - 



1 , nge^ 1 



4 2 



ln(C<5) / 



q{pi,<P,M {pi,,p) < 



ln(C<5)| 



n£ 



For this purpose, we let uj e |p£ <p — £, {\pe 



_ 11 2£\-^ > i _|_ ^es 
2 I 3 / — 4 ^ 21n(C5) 



to show pe < p, A4 {pe,p) < Clearly, pi < p follows immediately from pi < p — £■ To show 



I , = pg{Lu) and proceed 



(Pe^p) ^ ^^^^^> need to establish 



„ 1 2eV 1 n^e^ 



(16) 
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based on 



It is obvious that (fT6]) holds if j 



Pi 



2£ 

y 



^ 1 nie^ 

> T + 



(17) 



4 21n(C<5) 

^ ^^^l^Qg^ < 0. It remains to show (fTUj) under the condition that 



21n(C5) 



> 0. Note that p!7|) imphes either 



Pi 



2e 



> 



or 



1 









/I n^g2 
4 ^ 21n(C<5) 



1 ^ n^e^ 



4 21n(C(5) 



(18) 



(19) 



1 I 2e 



Since ([HD imphes either p,-i + f > ^ + ^/-. + ^^ > w| + or p,- ^^T - ^ 2Mc3): 



< 



> 

it must be true that imphes (fT6]) . On the other hand, (fT9]) also implies (fT6]) because ([19 

^ ' Hence, we have established based on (fT7|) . 



imphes ^/i + < 
1 

'2 



^ < - 2 + 3 

1 
2 



Since — ^<p^ — ^ + + have 5 — (p^ — | + > and, by virtue 



of dm), 



(P£ 



2^3; 



< 



ln(C<5) 



Hi 



Since < p — e, we have < < + e < p < 1. Hence, using the fact that M.{z,fi) is 
monotonically decreasing with respect to /i G (z, 1) as asserted by Lemma [U we have {p£,p) < 
{ptiPn + e) < '^^^ proof of the lemma is thus completed. □ 



Lemma 12 {p,>p + e, = 1} c |p, > p, X (p,,p) < M| /or ^ = 1, • • • ,s. 
Proof. Since {D( = 1} = - 5I - t)^ - i + 2\llls) }' suffices to show 



,Pi>P + £. 



Pi 



For this purpose, we let uj g > p + e, (|pf ~ 5I ^ - i + 2 infc^) } ' ~ Pf(^) ^'^'^ proceed 
to show p£ > p, Ai {p£,p) < ^^nP ■ Clearly, pi > p follows immediately from pi > p + e. To show 
■M. ipe,p) < ^^n^^ ) we need to establish 



1 2e\" 1 n^e" 
Pi - - - — ] > - + 



2 3 



4 21n(C(5) 



based on 



2e\^ 1 nie^ 

3 ■ + 



4 21n(C(5) 



(20) 



(21) 
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It is obvious that ([20]) holds if i 



21n(C5) 



21n(C5) 



< 0. It remains to show (1201) under the condition that 



> 0. Note that (j2ip imphes either 



Pe 



2e 



> 



' 1 n£e^ 
4 ^ 21n(C<5) 



or 



2e 



< 



4 21n(C5) 



(22) 



(23) 



or 



Since ([22]) imphes either p,-i-f < -f-^i + 2lKm< - V * ^ 2MC5) 2"- ^ y I ^ 2TK{cs) 
it must be true that (p2]) imphes ([20]) . On the other hand, (p3]) also implies ([20]) because ([23 
implies pg 



- ¥ < - 



Since 



1 



2 < p + e 
and, by virtue of ([20]) 



" 21n(C5) 
3 



. Hence, we have established (|20p based on ([2ip . 



2e 



< 1 



^ < ^, we have | 



2e^ 



> 



< 



HC5) 



Since pe > p + e, we have < p < — e < < 1. Hence, using the fact that Ai{z,fi) is 
monotonically increasing with respect to n G (0, z) as asserted by Lemma [H we have ipe,p) < 
{Pi^Pe — £) ^ ^^ni^ ■ '^^^ proof of the lemma is thus completed. 

□ 



Now we are in a position to prove Theorem 1. Since 24e-i6£'^ ^ ^ ^^■^ ^ ^ I-'' have 
r > 0. Hence, the sequence of sample sizes rii, ■ ■ ■ , is well-defined. By Lemma [TO] the sampling 
must stopped at some stage with index £ E {!,••• ,s}. This shows that the sampling scheme is 
well-defined. Noting that {n = n^} C {Dg = 1} for £ = 1, - ■ ■ , s, we have 

s s 

PT{\p-p\>e} = ^{pe<p- e, n = ne} + ^{p(,>p + e, n = ni} 

1=1 e=i 

s s 

< ^{p, <p-e, -Df = l} + 5^{p,>P + e, £>^ = 1}. (24) 



By Lemmas 1 1 1 1 and [9l 



^{p, <p-e, De = l}<j2{pi<P, M ipe,p) < < sCS < (r + 1)C<5. 

1=1 i=i ^ ) 



By Lemmas 1121 and [8] 



^{p,>p + e, D, = l]<Y,{p,>p, M{p,,p) < ^^^\ <sC5<{t + 1)C6. 
e=i i=i ^ "^e J 



(25) 



(26) 



Combining ([2l]) . ([25]) and ([26]) yields Pr{|p — p| > e} < 2(r -|- 1)C5- Hence, if we choose C to be a 
positive number less than 2{t+i) ' have Pr{|p — p| < e} > 1 — 5. This completes the proof of 
Theorem 1. 
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C Proof of Theorem 2 



Throughout the proof of Theorem 2, we define 

■ Pe ] — Pi 

= mm - Ea, > , Pi= max + ^ _ ^ 

By tedious computation, we can show the fohowing lemma. 

Lemma 13 For i = 1, - ■ ■ ,s, 

^ 6(l + £.)(3 + £..)ln(C<5) ) r / \ \niC6)\ 



6(l-e,)(3-£,)ln(C^) ] _ [ < MC^\ 

2(3 - £,)2 ln(C(5) - 9n,£2 / \-' 1^^^' 1 - e J " / 



Lemma 14 {p^ > c {p, > p, X < }. 



Proof. To prove the lemma, we let to £ {p^ ^ p}, Ps = Psi^)^ Pg ~ Pg^'^^ ^'^'^ proceed to 
show Ps > p, M ips,p) < ^^v^- Clearly, ps > p follows immediately from > p > 0. To show 
{Ps,p) < ^'^n^^ 1 shall first show M{pstpJ < ^^v^' simplicity of notations, we denote 
p* = —. We need to consider three cases as follows. 



Case (i): % < p* — Sa- In this case, 

MiPs,p)^M {ps^Ps -ea)<M {ps.Ps + ea)<M {p* - Ea^p*) < M {p* + ea,p*) < i^^Mi. 
Here the first inequality is due to Sa < p + £a ^ P + £a = Ps ^ P* — £a < and the fact that 



Ai{z, z + e) > Ai{z, z — e) for e < z < ^, which is asserted by Lemma [2j The second inequality is 
due to Ea < p + Ea < p ^ + £a = Ps < P* — < \~^a and the fact that M.{z,z + £) is monotonically 
increasing with respect to z E (0, ^ — e), which can be seen from Lemma [T] The third inequality 
is due to £a < p* < \ and the fact that M.{p + e^p) > A4{p — £,p) for e < p < ^ as a result 
of Lemma [H The last inequality is due to the fact that 
follows from the definition of n,. 



M{p*+ea,P*) 



> T^fT-T^r-^— TT , which 



Case (ii): p* — £a < Ps < P* + £a- In this case, 

M{ps,p)=M{ps,Ps~ea) < M {p" + ea,P* + Sa - Sa) = M {p* + Ea.p") < ^^^^ 

where the first inequality is due to £a < p + £a ^ P^ + £a = Ps < P* + £a < ^ — ^ + £a and the 
fact that M{z, z — e) is monotonically increasing with respect to z G (e, ^ + ^), which can be 
seen from Lemma [TJ 

Case (iii): ps > p* + £a- In this case, 

M{ps,pJ^M(ps,-^)<M(p* + Sa,^^^) =Mip* + ea,pn<^^- 
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where the first inequahty is due to the fact that A4{z, z/{l + e)) is monotonically decreasing with 
respect to z G (0, 1), which can be seen from Lemma [5l 

Therefore, we have shown M{ps,pJ < '"^'''^^ for all cases. As a result of Lemma [6l M{z,fi) 
is monotonically increasing with respect to /x G {0,z). By virtue of such monotonicity and the 
fact that Ps > > p > 0, we have M {ps,p) < M{ps,pJ < This completes the proof of the 

lemma. 

□ 



Lemma 15 {p, <p}c {p^ < p, M {p„p) < i^^}. 

Proof. To prove the lemma, we let lo G {p^ < p} , Ps = Pg{uj), Pg = Psi^) and proceed to 
show Ps < p, M {ps,p) < '"^^"^^ . Clearly, ps < p follows immediately from p^j < p < 1. To show 
M {ps,p) < ^^^^^) we shall first show M {ps,Ps) < ^^n^'' by considering three cases as follows. 

Case (i): ps <p* — £a- In this case, 

M {Ps,Ps) = M {ps,Ps +ea)<M (jp* - ea,p* -ea + ea)=M {p* - ea,p*) < X (p* + ea,p*) < i^^Ml . 

Tig 

Here the first inequality is due to < ps < p* — £a < ^ — £a and the fact that ^A{z,z + e) is 
monotonically increasing with respect to ^ G (0, ^ — e), which is asserted by LemmafU The second 
inequality is due to Sa < p* < and the fact that ^A{p + e,p) > A4{p — £,p) for e < p < ^, which 
can be seen from Lemma HI 

Case (ii): p* — Sa < Ps < P* + £a- In this case, 

M {ps,Ps) = M (^ps, <m(^p*- ea, ^M{p*- ea,pn < M {p* + e^.p^) < 

where the first inequality is due to < p* — Ea < Ps = (1 — er)Ps ^ (1 — ^r)p < I — £r and the fact 
that M{z, z/{l — e)) is monotonically decreasing with respect to z G (0, 1 — e), which is asserted 
by Lemma [5l 

Case (iii): ps > p* + £a- In this case, 
M ips,pg) = m(ps, <m(ps, <M(p*+Sa, =M{p* + ea,p*) < 



Here the first inequality is due to < = {1 — £r)Ps < {1 — £r)p < I — £r and the fact that 
M{z,z/{l + £)) > A4{z, z/{l — £)) for < z < 1 — e, which can be seen from Lemma[3l The second 
inequality is due to p* + £a ^ Ps and the fact that M.{z,z/{1 + e)) is monotonically decreasing 
with respect to z G (0, 1), which is asserted by Lemma [5l 

Therefore, we have shown M (PsiPs) < ^'^n^^ all cases. As a result of Lemma El A4{z,^) 
is monotonically decreasing with respect to /.i G (z, 1). By virtue of such monotonicity and the 
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fact that < ^ y < li we have M {ps,p) < M iPs,Ps) < ^'^f^'' ■ This completes the proof of the 
lemma. 

□ 



Lemma 16 {p^ < p, -Df = 1} c < p, M {Pi,p) < forl<£<s. 



Proof. To show the lemma, we let to £ {pi < p, Di = 1}, pi = Piiuj), pi = Pe{io) and proceed 
to show pe < p, M {pi,p) < ^'^nP ■ Clearly, pi < p follows immediately from P£ < p < 1. To show 
■M. ip£,p) < '"^^"^^ , we shall first show M (pe^Pi) < '"^^"^^ by considering three cases as follows. 



Case (i): pe < — — £a- In this case, by the definition of the stopping rule, we have pi <\ 



2Ea 



4 ' 21n(C<5) ' 

we have 



Hence, | 



which implies j 



[Pt 



1 2ea _ 

2 + — =^^ 



+ 



21n(Ci5) 

2ea 



Observing that 



+ ea <p <l, 



< 



2 3 2 



+ %^)^ > and M {pi 



< 



ln(C<5) 



Case (ii): 



Pe 



< £a- In this case, by the definition of the stopping rule and (|28p of Lemma 



[T3l we have M. {pe,Pe) < 



nt 



+ea. 



withp^ = 

In this case, we have 



and M {pe,Pe) = M (^pe, 



Case (iii). pe ^ i ^a- j-^^ uiiio <^h,j\j, wv. liuiw, — i—er "^^^ ^-^ xnii^u — i/^ti i—^^ 
(p^i T+^) — ^"^^/^ • Here, the first inequality follows from Lemma [3] and the fact that 



< 



The second inequality follows from the definitions of the stopping 



(1 - er)pe < (1 - er)p < 1 - er 
rule and (|27p of Lemma [T3j 

Therefore, we have shown M{pe,P() < ^^^^^ for all three cases. As a result of Lemma [6l 
A4{z,fi) is monotonically decreasing with respect to ;U G (z, 1). By virtue of such monotonicity 
and the fact that < pi < p^ < p < 1, we have M {p£,p) < M {pt^Pi) < This completes 

the proof of the lemma. 

□ 



Lemma 17 {p, > p, = l} C {p, > p, M {p,,p) < for 1 < I < s. 

Proof. To show the lemma, we let cu G {p^ > p, Di = 1}, p£ = Pi{uj), p^ = p^{lo) and proceed 
to show Pe > p, M {pe,p) < ^^^n^ ■ Clearly, pi > p follows immediately from p^ > p > 0- To show 

{pe,p) < ^'^nP 1 we shall first show M{pi,Pj) < ^^7^ by considering three cases as follows. 

Case (i): pi < — £a- In this case, we have p^ = pe — £a and 

£a < P + ea < P£ + ea = Pi < < 7^ ^, 

where the last inequality follows from the assumption about and e,.. By virtue of the fact that 
<Pi < \ - and Lemma [21 we have M(j)uPj) = M {pe,Pe - ea) < M ipe,pe + Sa). Since 
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< Ea - ^ + ^ < pe - ^ + ^ < ^, we have I - {pe - ^ + ^) > 0. By the definition of the 
stopping rule, we have Pi<^~^- \[k^r^^, which imphes \-{pi-l + < - 2\^(C5) ^'^'^ 
thus M{n,p,) < M ipi,pi + < i^M. 



Case (ii): 



< Ea- In this case, since p^ = P£ — Ea > p > 0, we have 

1 1 2Ea 1 2Ea _ 1 2^^ 1 

-^<P + e,-^- — <P, + ea--- — = Pi- -- — <-.. 



which imphes | — [pi — ^ — > 0. By the definition of the stopping rule, we have p£ < 

5 + ¥ - \/l + Jltm^ which implies i - - i - < -JIL^. It follows that Mipe,p^) - 

M - sa) < ^ because \ - {Pi - k - ^ f > 0- 

Case (iii): Pi > ^ + Ea- In this case, by the definition of the stopping rule and (j27p of Lemma 
da we have M{pi,p^) < with p^ = 

Therefore, we have shown M{pe,p^) < '"^^"^^ for all three cases. As a result of Lemma [6l 
M{z,fi) is monotonically increasing with respect to p, £ (0,z). By virtue of such monotonicity 
and the fact that < p < p^ < pi < 1, we have M. (pi,p) < M.iPi,Pj) < ^^n^^ ■ This completes 
the proof of the lemma. □ 

Now we are in a position to prove Theorem 2. As a direct consequence of the assumption 
that < Ea < I and -^z^f- < £r < 1, we have | (^j — — > 1, which implies that r > 0. This 
shows that the sequence of sample sizes ni, • • • , is well-defined and it follows that the sampling 
scheme is well-defined. Invoking the definitions of p^, and noting that {n = n^} C {Di = 1} 
for ^ = 1, • • • , s, we have 

s 

Pic{\p - p\ > Ea, \P - p\ > ErP} = '^Pv{\pe - p\ > Ea, \Pe - p\ > ErP, n. = Ui} 

e=i 

s s 

= ^{Pe -P^ n = ni} + ^{p^ > p, n = n^} 

e=i 1=1 

s s 

< ^{Pe <P, De = l} + ^{p, >p, De = 1}. (29) 



e=i 



By Lemmas [TBI ESI and [9l 



^{p, <p,De = l}<j2 {pi <P'^ (Pi^P) < < <^ < + 1)C<5- 

£=1 1=1 ^ J 



(30) 



By Lemmas [HI [H and El 



Y,{P, >P,De = l}<j2 {pi >P^M {p,,p) < < sC5 < (r + 1)C<5. (31) 

£=l 1=1 ^ "-^ J 

Combining ([29]) . ([30]) and ([3T]) yields Pr{|p — p\> Ea, \p — p\ > £rP} < 2(r + 1)C<^- Hence, if we 
choose C to be a positive number less than 2[t+i) ' have Pr{|p— p| < Ea or |p— p| < Erp} > 1 — 6. 
This completes the proof of Theorem 2. 
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D Proof of Theorem 3 



In the course of proving Theorem 3, we need to use the following lemma regarding inverse binomial 
sampling, which has been established by Chen in [1]. 

Lemma 18 Let Xi,X2, ■ ■ ■ be a sequence of i.i.d. Bernoulli random variables such that Pr{Xi = 
1} = 1 — Pi{Xi = 0} = p G (0, 1) for i = 1,2, • • • . Let n be the minimum integer such that 
Y^^^i Xi = J where ^ is a positive integer. Then, for any a > 0, 

Pr|- <P, (-,p) < — } <a, Pr(- >P, ^^i f-,p) < — j < a 

where 

flnf + (i-l)lnief /orze (0,1) and/xG (0,1), 
^i(2;,/i) = < In^ for z ^ I and ^ e {0,1), 

[ —00 for z = and /.j G (0, 1). 

Lemma 19 Let Xi,X2, ■ ■ ■ be a sequence of i.i.d. Bernoulli random variables such that Pr{Xj = 
1} = 1 — Pr{Xj = 0} = p G (0, 1) for i = 1, 2, • • • . Let n be the minimum integer such that 
= 7 where ^ is a positive integer. Then, for any a > 0, 

<p, <i^| <a, (32) 

7 \ In a 1 , , 

-,p)< \<a 33 

n / 7 J 

where Mi{z, /i) = MiliEl forO<z<l andO< ^i<l. 

Proof. By Massart's inequality (i.e., Theorem 2 at page 1271 of [8]), we have ^i{z,p) < 
A4i{z,p) for any z G (0,p). By virtue of this fact and Lemma [TSl we have 

7 \ In a 1 

-,p) < ^ <«, 

n / 7 J 





'7 






Pr< 









1 


< 


P, 


Mi\ 


(1 


,p) < 


In a ] 


j> < Pr <j 


r7 


< 


p, 


n 








\n 




7 J 




[n 






1 


> 


P, 


Mi\ 


a 


,p) < 


In a ] 


j> < Pr <j 


\l 


> 


p, M\ 


n 












7 J 




{n 







7 „\ ^ In a 
n 



< a. 



This completes the proof of the lemma. 



□ 



In the sequel, we define random variables Di, I = I,-- - ,s such that Di = 1 if 7^ > 
2(3+e)2'^inf^"!5)-9e2n|, and Di = othcrwisc. Then, the stopping rule can be restated as "sampling 
is continued until = 1 for some i £ {I,-'' j*}"- For simplicity of notations, we also define 
Do = 0. 

By tedious computation, we can show the following lemma. 
Lemma 20 = 1} = {a^i (pi, ^) < '-^} for £ = 1, ■ ■ ■ , s. 

Lemma 21 {p,<p{l-e), De = l} C^p,<p, Mi (p^p) < '-^j for i = 1, ■ ■ ■ ,s. 
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Proof. Let w £ {p^ < p{l — e), De = 1} and = Pe{i^)- To show the lemma, it suffices to show 
Pe<p and Mi {pe,p) < ^f^-. By Lemma [201 



{p, < p{l - e), De = l} = ^Pe< p{l - e), Mi ( p„ - 



Pi \ ^ HCS) 



which imphes pi < p{l — e) and Mi ipe, < . Clearly, pi < p{l — e) implies pe < p. To 



show Mi{pe,p) < '"^"^"^^ , we shall consider two cases as follows: 



ln(C-?) 

In the case of pi > 0, we have < < p{l — e) < 1 — e. Since 



In the case pe = 0, we have Mi {pi,p) = — oo < 



l^' TT7 j - l^' j ^ 3(1 + 1) + .(!-!)] (i-|)[i-.-.(i-t)] > « 

for < ^ < 1 - e, we have TWi (p^, < TWi (p^, < ^i^^. Note that 

dMi{z, fi) _ {z- fi) - z) + z{l - Ai) + 2(1 - z)] 
3-[(f + !)(!- f- I)]' ' 
from which it can be seen that Mi{z,^) is monotonically decreasing with respect to /U G (z, 1). 
By virtue of such monotonicity and the fact that < pe < < p < 1, we have Mi{p£,p) < 
■Ml (pe^ 1^) < '"^^"^^ ■ This completes the proof of the lemma. 



□ 



Lemma 22 {p, > p{l + e), = 1} C {p, > p, Mi {p,,p) < ^} for £ = 1, 



, s. 



Proof. Let lo e {pi > p{l + e), = 1} and pe — Piito)- To show the lemma, it suffices to show 
Pi>p and Ml {pe,p) < By Lemma [20l 

{p, > p{l + e), D, = l} = ip, > p{l + e), Ml (pf, < ^""^^^^ ^ 



1 + eJ It } 

which implies pi > p{l + e) and Mi {pi, < '"^"^"^^ . Clearly, pn > p{l + e) implies pi > p. Since 
1 > > p(l + , we have < p < ^ < < 1 . Noiing that = > 

for < /I < z < 1, we have Mi {pt,p) < Mi {pe, -^"j < This completes the proof of the 

lemma. 

□ 



Lemma 23 Ds = 1. 



22 



Proof. To show Ds = 1, it suffices to sliow Mi yz^j^j < ' for any z G (0,1]. Tliis is 

because < < 1 for any lo ^ Q, and {Ds = 1} = |a^i ^p^, < '"^^^ | as asserted by 

Lemma [20l 



By tlie definition of sample sizes, we have 7s 



> T^- Since 



lim.^o Mi{z,j^)= [2 (1 + §)(! + e)] ' < 0, we have lim.^o Mi{z,j^) < 



e2[2(l+f)(l+£) 

Note that A^i ^z, = — ^^^i^e^j^^g.^^, e i from which it can be seen that Mi (^z,j^^ is 
monotonically decreasing with respect to z G (0,1). Hence, Mi (^z,jj^^ < lim2^o-^i < 
'"^^^ for any z G (0, 1). Since Mi ^z, is a continuous function with respect to z G (0, 1) and 
Ml (l, = lim,,^i Ml (2, , it must be true that Mi (l, < ^^K*). This completes the 
proof of the lemma. 

□ 

Lemma 24 Pr{p < p{l-e)} < Y.i=i^^ {Pi ^ P(l -^)' -^^^-i = 0, De = 1} < {t + l)Q5 for any 
pG (0,1). 

Proof. By Lemma [23l the sampling must stop at some stage with index I G {1, • • • , s}. This 
implies that the stopping rule is well-defined. Let 7 = X^JLi^i- Then, we can write Pr{p < 
p{l — e)} = J2e=i^''^{Pe - Pi^ ^ 7 = 7^}- By the definition of the stopping rule, we have 
{7 = li} ^ {De-i =0, De = 1}. Hence, 

S S 

Pr{p<p(l-£)}<^Pr{p, <p(l-e), 0, £>, = 1} < ^ Pr {p, < p(l - e), = (34) 

e=i 1=1 

Applying Lemma ED and ([32]) of Lemma [T9t we have 

^Pr{p, <p(l-£), r>f = l}<^Pr|p, <p, Xi(p„p)< <<5< (r + l)C<5. (35) 

Finally, the lemma can be established by combining (j34p and (j35p . 

□ 

Lemma 25 Pr{p > p{l + e)] < Y.Ui'^^iPe > P(l + ^^-i = 0, 1)^ = 1} < (r + 1)C<5 /or ant/ 
pG (0,1). 

Proof. Note that 

S S 

Pr{p>p(l + £)} < ^Pr{p, >p(l + e), De-i ^ 0, £>, = 1} < ^ Pr {p, > p(l + e), = (36) 
Applying Lemma [22] and (|33|) of Lemma [T9l we have 



^Pr{p, >p(l + £), Dc = l}<;^Pr|p, >p, Xi(p„p)< 

£=1 £=1 7f J 



< sC5 < (r + 1)C<5. (37) 
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Combining (j36p and (j37p proves the lemma. 

□ 

Finally, we are in a position to prove Theorem 3. Noting that Pr{|p — p\ > ep} = Pr{p < 
p{\ — e)} + Pr{p > p(l + e)} and making use of Lemmas 1241 and 125^ we have Pr{|p — p\'> ep} < 
(t + 1)CS + (r + = 2(r + 1)(6 for any p £ (0, 1). Hence, if we choose C to be a positive 
number less than 2{t+i) ' have Pr{|p — p\ > ep} < S and thus Pr{|p — p\ < £p} > 1 — S for any 
p £ (0,1). This completes the proof of Theorem 3. 
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